Front end applications are computer programs which typically provide a Graphical User Interface (GUI) through which a user may interact with the application. The use of a GUI is well established in the computing field, and is used in the control of operating systems, text editing applications and calculation applications among many others.
The GUI of an application is designed to show a number of screens. A screen may comprise a window or dialogue box displayed on a part of a visual display unit (VDU), or alternatively a screen may occupy the whole of a VDU. These screens each contain a set of GUI objects; a GUI object is anything in the screen which may be interacted with by the user. For example, a screen of a front end application may comprise the “Save As . . . ” dialogue box, in which the “Save” button, “Cancel” button, “Save As Type” drop down menu, “File Name” text entry box all constitute GUI objects. In addition, the screen itself, i.e. the dialogue box, constitutes a GUI object which contains parameters such as the name, class, size and position of the screen (dialogue box).
When an application is being developed it is customary to test the application to ensure that the application is working correctly. This testing is intended to uncover defects or “bugs” in the application. The defects may arise from both architectural error (that is an error in the specification of the application) and programming error (a mistake in the writing of the application's computer code).
Any such test will emulate a user interacting with the GUI objects of the application under test (AUT). To do this the test will define a series of actions to be performed on the GUI objects and the expected results of such actions. To give an example, a test may define that the close button of a particular window be clicked, the expected result being the closing of the window.
A first type of application test is called a manual test. A manual test is derived from the AUT's requirements and will be written by one or more of a manual tester, a Subject Matter Expert (SME), or a Business Analyst (BA) who has knowledge of the AUT. To create a manual test a series of user actions and verification steps are documented in chronological order. These manual test actions may include specifying that a particular button is to be clicked using the mouse, or that certain text is to be inputted on the keyboard. To test an AUT using a manual test, a human tester has to perform each step as if that tester were a user. In addition the tester has to confirm that the expected results defined in the verification steps have been achieved. A manual test has the disadvantage that the tester has to be present to perform the test, increasing the cost and time of performing a test on an application.
A further type of application test is an automated test. In an automated test, the actions and verifications of the manual test are performed by a computer emulating the human tester. Automated tests are normally derived from manual tests; that is, a manual test is defined by the manual tester, SME and/or BA and then the steps of the manual test are used by a programmer as a basis for creating the automated test. The process of creating an automated test is more laborious than a manual test, and consequently not all manual tests are converted into automated tests.
An automated test script may be created using an automation technology. Each automation technology may use a scripting language to describe the recorded steps in a computer understandable form. Examples of such automation technologies and their scripting languages (in brackets) are:
Hewitt Packard, QuickTest Professional(Visual Basic Script)IBM Rational, IBM Rational(Java or Visual Basic)Functional TesterIBM Rational, Rational robot(Java or Visual Basic)Micro Focus, SilkTest(Visual Basic, .Net or C#)
To create an automated test from a manual test using an automation technology the following operations need to be performed:
Baselining
The tester performs the actions on the AUT defined in a manual test while the automation technology ‘records’ these steps. In recording the steps, the automation technology generates a ‘baseline script’ of computer code. The code in the baseline script uniquely identifies a GUI object in a screen and identifies an action to be performed on this object. The baseline script is then modified, as described below, to form the final automated test script.
Specifying Synchronization Points
When a given test action is performed, the GUI displayed by the AUT will change in response to the test action. This change must occur before any subsequent test action is performed. For example, when a “Login” button is clicked (assuming correct login details), an AUT may be programmed to replace a login window with a welcome screen. When the next action to be performed by the test is to an object in this welcome screen, the test must first ensure that the AUT is displaying the correct screen and is therefore ready for this next action; that is, the test needs to ‘synchronize’ with the AUT. During a manual test, the human tester will instinctively wait for the new screen to appear before attempting the next action—a human tester will instinctively synchronize with the AUT. However a computer running an automated test script needs to be programmed to do so.
Consequently, synchronization points are defined within the test script code. To do this, the baseline script is replayed and it is determined where the automated test script needs to perform synchronizations. The code within the baseline script is then modified to include synchronization code which explicitly defines the synchronization requirements. Typically, the synchronization code will define a period during which the test waits, and a result (such as the appearance of the welcome screen) which is to be tested for during that time.
Defining Verification Checkpoints
The manual test defines the expected result of each action. Corresponding verification code therefore needs to be included into the automated test script. The verification code defines verification checkpoints which are points in the execution of the code at which a comparison between the expected response of the AUT and the actual response of the AUT is made. The verification code defines both the test, and code to report both success and failure.
Script Testing
The final automated test script is replayed to check that the script code is of valid syntax and will provide valid responses for both pass and failure conditions of the application.
As can be seen, the process by which a manual test is converted into an automated test is laborious. As a result, not all manual tests are automated. In addition, new versions of the AUT will be periodically released. Each new version will be a modified version of the previous one. The modifications may correct defects; alternatively the modifications may add new features to the AUT. Each new version of the AUT may require an automated test script to be re-recorded (with the corresponding steps as defined above). This can further increase the labour required to test an application.
In addition, any changes to a test script need to be defined within the scripting code. This requires specialist knowledge of the scripting code being used, knowledge which may not be available to a subject matter expert defining the test. Consequently, an expert in scripting is required to bridge this gap.
Therefore there is a need for a system in which manual tests can be readily automated. There is a further requirement for a system in which an automated test can be readily edited by a person lacking expert knowledge of scripting systems. Moreover, there is a need for a system in which existing automated tests can be easily updated to reflect the modifications to the AUT.